Assignment 5: Probabilistic Learning - Classification Using Naive Bayes
Lau Wen Jun
2023-04-12
1 Explain these two concepts
Bayes Theorem
- A theorem describing how the conditional probability of an event, based on the occurrence of another event, is equal to the likelihood of the second event given from prior knowledge of conditions that might be related to the event. Bayesian classification uses Bayes theorem to predict the occurrence of any event. Classification Tasks is generated by a supervised machine learning algorithm known as the Naïv Bayes classifier
Laplace Estimation
- Laplace estimation tackle the problem of zero probability that make the entire calculations as zeros, through estimator by adding a small number to each of the counts in the frequency table, which ensures that each feature has a nonzero probability of occurring with each class.
2 Processing text data for analysis
Load the SOCR
2011 US Job Satisfaction data. The last column
(Description) contains free text describing each job type.
Notice that spaces are replaced by underscores, __. To mine
the text field and suggest some meta-data analytics, construct an R
protocol for:
Convert the textual meta-data into a corpus object.
library(xml2) library(rvest) library(tm)js<-read_html('http://wiki.socr.umich.edu/index.php/SOCR_Data_2011_US_JobsRanking#2011_Ranking_of_the_200_most_common_Jobs_in_the_US') html_nodes(js, "#content")## {xml_nodeset (1)} ## [1] <div id="content" class="mw-body" role="main">\n\t\t\t<a id="top"></a>\n\ ...js<- html_table(html_nodes(js, "table")[[1]])js_corpus<-Corpus(VectorSource(js$Description)) js_corpus## <<SimpleCorpus>> ## Metadata: corpus specific: 1, document level (indexed): 0 ## Content: documents: 200inspect(js_corpus[1:3])## <<SimpleCorpus>> ## Metadata: corpus specific: 1, document level (indexed): 0 ## Content: documents: 3 ## ## [1] Researches_designs_develops_and_maintains_software_systems_along_with_hardware_development_for_medical_scientific_and_industrial_purposes ## [2] Applies_mathematical_theories_and_formulas_to_teach_or_solve_problems_in_a_business_educational_or_industrial_climate ## [3] Interprets_statistics_to_determine_probabilities_of_accidents_sickness_and_death_and_loss_of_property_from_theft_and_natural_disastersjs_corpus1<-Corpus(VectorSource(js$Job_Title)) js_corpus1## <<SimpleCorpus>> ## Metadata: corpus specific: 1, document level (indexed): 0 ## Content: documents: 200inspect(js_corpus1[1:3])## <<SimpleCorpus>> ## Metadata: corpus specific: 1, document level (indexed): 0 ## Content: documents: 3 ## ## [1] Software_Engineer Mathematician ActuaryTriage some of the irrelevant punctuation and other symbols in the corpus document, change all text to lower case, etc.
corpus_clean<-tm_map(js_corpus, tolower)## Warning in tm_map.SimpleCorpus(js_corpus, tolower): transformation drops ## documentscorpus_clean<-tm_map(corpus_clean, content_transformer(function(x) gsub(x, pattern = "_", replacement = " ")))## Warning in tm_map.SimpleCorpus(corpus_clean, content_transformer(function(x) ## gsub(x, : transformation drops documentscorpus_clean <- tm_map(corpus_clean, removeWords, stopwords("english"))## Warning in tm_map.SimpleCorpus(corpus_clean, removeWords, ## stopwords("english")): transformation drops documentsinspect(corpus_clean[1:3])## <<SimpleCorpus>> ## Metadata: corpus specific: 1, document level (indexed): 0 ## Content: documents: 3 ## ## [1] researches designs develops maintains software systems along hardware development medical scientific industrial purposes ## [2] applies mathematical theories formulas teach solve problems business educational industrial climate ## [3] interprets statistics determine probabilities accidents sickness death loss property theft natural disasterscorpus_clean1<-tm_map(js_corpus1, content_transformer(function(x) gsub(x, pattern = "_", replacement = " ")))## Warning in tm_map.SimpleCorpus(js_corpus1, content_transformer(function(x) ## gsub(x, : transformation drops documentsinspect(corpus_clean1[1:3])## <<SimpleCorpus>> ## Metadata: corpus specific: 1, document level (indexed): 0 ## Content: documents: 3 ## ## [1] Software Engineer Mathematician ActuaryTokenize the job descriptions into words. Examine the distributions of
Stress_CategoryandHiring_Potential.js_dtm<-DocumentTermMatrix(corpus_clean) js_dtm## <<DocumentTermMatrix (documents: 200, terms: 1066)>> ## Non-/sparse entries: 1832/211368 ## Sparsity : 99% ## Maximal term length: 16 ## Weighting : term frequency (tf)prop.table(table(js$Stress_Category))## ## 0 1 2 3 4 5 ## 0.060 0.425 0.320 0.120 0.065 0.010prop.table(table(js$Hiring_Potential))## ## -40.76 -31.88 -30.29 -24.06 -20.18 -19.57 -17.68 -16.96 -15.74 -14.5 -14.37 ## 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## -14.31 -14.25 -14.14 -13.62 -12.78 -12.73 -12.62 -12.35 -11.43 -11.36 -11.25 ## 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## -11.22 -10.53 -10.48 -9.33 -9.32 -9.28 -8.53 -8.37 -8 -7.86 -7.24 ## 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## -7.09 -6.78 -6.75 -6.53 -6.4 -5.5 -5.25 -5.07 -5 -4.17 -3.93 ## 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## -3.71 -3.68 -3.66 -3.59 -3.58 -3.54 -3.53 -3.48 -2.93 -2.91 -2.89 ## 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## -2.68 -2.65 -2.59 -2.58 -2.47 -2.05 -2 -1.61 -1.25 -1.24 -0.95 ## 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## -0.92 -0.84 -0.81 -0.69 -0.6 -0.44 -0.17 -0.16 0 0.09 0.5 ## 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## 0.53 0.67 0.69 0.79 1 1.07 1.08 1.18 1.19 1.36 2.09 ## 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## 2.28 2.6 2.65 2.84 3.35 3.39 3.64 3.82 4.18 4.27 4.38 ## 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## 4.5 4.68 4.7 5.09 5.19 5.22 5.27 5.39 5.45 5.53 5.66 ## 0.005 0.005 0.010 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## 5.81 5.93 6.22 6.35 6.45 6.48 6.52 6.65 6.79 6.86 7.11 ## 0.005 0.005 0.005 0.005 0.005 0.010 0.005 0.005 0.005 0.005 0.005 ## 7.19 7.54 7.69 7.71 7.85 8.14 8.3 8.32 8.43 8.56 8.74 ## 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## 8.79 8.86 9.42 9.49 9.5 9.71 10 10.1 10.11 10.14 10.38 ## 0.005 0.005 0.010 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## 10.47 10.89 11.03 11.08 11.78 11.96 12.1 12.14 12.76 12.85 13.63 ## 0.005 0.005 0.005 0.010 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## 13.97 14.26 14.29 14.33 14.44 14.7 14.74 15.11 15.48 15.53 15.69 ## 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## 16.57 16.74 17.04 17.43 17.54 17.55 17.56 17.65 18.22 18.9 19.11 ## 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## 19.63 19.78 19.93 20.14 20.58 20.65 21.29 21.44 21.58 22.39 23.53 ## 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 ## 24.1 24.22 25.75 25.9 27.4 28.04 32.98 33.07 37.05 ## 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005 0.005Classify the job Stress into two categories.
js2<-cbind(js,corpus_clean$content,corpus_clean1$content) js2$`corpus_clean$content`->js2$cleandesc js2$`corpus_clean1$content`->js2$jt js2<-js2[,c(-2,-11,-12)] js2## Index Overall_Score Average_Income(USD) Work_Environment Stress_Level ## 1 1 60 87140 150.00 10.40 ## 2 2 73 94178 89.72 12.78 ## 3 3 123 87204 179.44 16.04 ## 4 4 129 73208 89.52 14.08 ## 5 5 147 77153 90.78 16.53 ## 6 6 175 85210 179.64 15.10 ## 7 7 182 74278 314.37 15.78 ## 8 8 192 63208 136.41 17.08 ## 9 9 195 63144 463.43 9.44 ## 10 10 197 67107 593.25 12.07 ## 11 11 200 70122 322.24 19.47 ## 12 12 216 60174 276.78 19.74 ## 13 13 217 47153 263.82 16.53 ## 14 14 217 106196 269.46 16.96 ## 15 15 223 101164 225.00 18.64 ## 16 16 225 61221 361.44 12.56 ## 17 17 230 70193 505.56 13.22 ## 18 18 236 47155 381.06 12.55 ## 19 19 240 95130 230.30 20.30 ## 20 20 247 87240 227.35 17.40 ## 21 21 251 65143 463.43 12.43 ## 22 22 252 105233 314.37 21.33 ## 23 23 256 55239 390.56 14.11 ## 24 24 257 109147 322.42 18.93 ## 25 25 270 52127 334.40 10.27 ## 26 26 270 63170 221.25 18.70 ## 27 27 273 96163 538.34 16.63 ## 28 27 273 71176 265.08 11.76 ## 29 29 282 54197 385.20 11.48 ## 30 30 285 55100 371.70 16.00 ## 31 31 292 48256 361.44 18.56 ## 32 32 296 68358 587.28 13.58 ## 33 33 297 76139 322.42 22.39 ## 34 34 298 30110 318.29 15.10 ## 35 35 299 36129 454.30 9.29 ## 36 36 302 109070 788.64 15.70 ## 37 37 307 75122 230.30 20.22 ## 38 38 309 55165 230.85 19.65 ## 39 39 311 33138 204.40 12.38 ## 40 40 312 31148 165.20 7.48 ## 41 41 312 85119 432.99 25.19 ## 42 42 317 35176 404.10 11.76 ## 43 43 330 49214 449.10 17.14 ## 44 44 332 83086 275.00 21.86 ## 45 45 333 74104 716.21 14.04 ## 46 46 338 53171 556.40 14.71 ## 47 47 339 51352 274.68 20.52 ## 48 48 349 51181 359.28 16.81 ## 49 49 352 97109 322.42 32.09 ## 50 50 355 58175 228.10 16.75 ## 51 51 360 58200 177.00 25.00 ## 52 52 365 121065 940.59 22.65 ## 53 53 367 48211 428.00 21.11 ## 54 53 367 113439 655.98 21.39 ## 55 55 370 46211 737.85 20.11 ## 56 56 372 81274 449.10 23.74 ## 57 57 373 116249 783.04 18.49 ## 58 58 375 62229 863.93 32.29 ## 59 59 376 34200 316.40 8.00 ## 60 60 380 53175 500.17 26.75 ## 61 61 390 46141 222.95 17.41 ## 62 62 392 77136 278.46 22.36 ## 63 63 395 33143 371.70 15.43 ## 64 64 398 51190 545.64 26.90 ## 65 65 399 81198 978.80 15.98 ## 66 66 402 33104 160.64 13.04 ## 67 67 404 23169 581.14 10.69 ## 68 68 410 43226 626.52 21.26 ## 69 69 411 68195 449.10 16.95 ## 70 70 413 33211 234.70 8.11 ## 71 71 422 57166 356.72 19.66 ## 72 72 423 54114 673.65 14.14 ## 73 73 424 40154 550.42 24.54 ## 74 74 428 45219 354.00 19.19 ## 75 75 431 142144 1119.75 23.44 ## 76 76 432 24169 575.12 8.69 ## 77 77 433 63145 538.92 17.45 ## 78 78 437 83135 318.92 23.35 ## 79 79 438 66179 727.52 28.79 ## 80 80 446 51246 486.75 25.46 ## 81 81 448 54279 752.25 29.79 ## 82 82 448 113211 1261.50 36.11 ## 83 83 449 192065 1471.50 25.65 ## 84 84 462 55142 493.79 23.42 ## 85 85 463 84105 1295.80 32.05 ## 86 86 469 24083 642.56 12.83 ## 87 87 477 44335 486.75 23.35 ## 88 88 482 53085 884.73 19.85 ## 89 89 483 31194 359.20 5.94 ## 90 90 491 26182 404.10 6.82 ## 91 91 495 23119 546.78 14.19 ## 92 92 496 160242 1417.00 24.42 ## 93 93 504 41154 633.50 20.54 ## 94 94 504 64114 1018.75 30.14 ## 95 95 506 40148 734.24 17.48 ## 96 96 506 46207 807.12 23.07 ## 97 97 506 58134 657.16 14.34 ## 98 98 507 44131 422.46 16.31 ## 99 99 518 33272 351.76 17.72 ## 100 100 528 51132 709.60 27.32 ## 101 101 528 365258 1962.00 30.58 ## 102 102 529 51132 657.16 16.32 ## 103 103 531 46342 1085.52 36.42 ## 104 104 531 25118 665.21 23.18 ## 105 105 533 18067 595.36 12.67 ## 106 106 536 62105 1010.10 41.05 ## 107 107 538 30147 469.40 11.47 ## 108 108 542 73193 1195.00 39.93 ## 109 109 542 32969 502.26 8.69 ## 110 110 547 37164 610.22 12.64 ## 111 111 548 67470 1155.75 39.70 ## 112 112 548 30114 454.50 10.14 ## 113 113 552 90160 1247.13 47.60 ## 114 114 556 161141 1540.00 47.41 ## 115 115 558 82210 923.21 31.10 ## 116 116 559 33229 563.28 14.29 ## 117 117 560 30147 504.36 10.47 ## 118 118 562 56132 610.22 24.32 ## 119 119 564 34150 494.01 19.50 ## 120 120 565 33165 440.90 16.65 ## 121 121 566 106153 1064.16 59.53 ## 122 122 571 34152 610.22 12.52 ## 123 123 573 38157 611.91 15.57 ## 124 124 574 54203 1015.40 29.03 ## 125 125 578 40357 1293.57 38.57 ## 126 126 581 31163 719.82 15.63 ## 127 127 583 40090 1080.00 25.90 ## 128 128 583 50456 929.25 43.56 ## 129 129 585 54197 956.97 27.97 ## 130 130 585 28132 520.32 14.32 ## 131 131 590 27147 704.31 7.47 ## 132 132 595 47122 740.55 14.22 ## 133 133 600 24135 812.06 16.35 ## 134 134 600 27263 639.45 8.63 ## 135 135 604 28150 639.45 9.50 ## 136 136 605 54145 830.28 32.45 ## 137 137 616 34122 753.10 14.22 ## 138 138 621 38138 756.80 13.38 ## 139 139 621 18100 687.42 15.00 ## 140 140 621 18107 796.74 13.07 ## 141 141 626 22125 489.60 15.25 ## 142 142 628 46182 836.38 22.82 ## 143 143 633 61291 929.25 36.91 ## 144 144 638 30265 663.75 28.65 ## 145 145 638 23112 597.24 17.12 ## 146 146 642 24089 872.96 22.89 ## 147 147 643 22138 889.14 20.38 ## 148 148 643 47176 855.76 20.76 ## 149 149 643 40184 803.00 44.84 ## 150 150 645 36186 796.86 26.86 ## 151 151 645 37163 821.88 34.63 ## 152 152 649 38178 669.90 18.78 ## 153 153 650 51308 1293.84 37.08 ## 154 154 651 31124 728.32 11.24 ## 155 155 655 53118 981.20 27.18 ## 156 156 661 49207 1108.08 37.07 ## 157 157 665 22119 945.80 21.19 ## 158 158 666 59222 1263.84 26.22 ## 159 159 666 20153 836.43 21.53 ## 160 160 673 39150 1646.75 31.50 ## 161 161 678 37179 973.94 19.79 ## 162 162 679 37238 796.86 18.38 ## 163 163 680 49116 1371.75 45.16 ## 164 164 681 51139 1055.47 27.39 ## 165 165 686 34167 1124.48 25.67 ## 166 166 692 53171 1733.40 40.71 ## 167 167 695 18053 868.32 14.53 ## 168 168 699 47175 1151.02 20.75 ## 169 169 700 19093 682.02 18.93 ## 170 170 701 43309 1293.57 36.09 ## 171 171 711 38128 968.20 28.28 ## 172 172 716 31147 1134.77 40.47 ## 173 173 720 40169 1136.25 23.69 ## 174 174 725 52042 1461.24 23.42 ## 175 175 729 39184 973.36 28.84 ## 176 176 730 45222 3314.03 60.22 ## 177 177 730 28375 1062.00 29.75 ## 178 178 731 49185 1877.85 43.85 ## 179 179 731 32174 1368.32 23.74 ## 180 180 738 38283 1106.25 38.83 ## 181 181 742 38160 929.67 24.60 ## 182 182 743 29150 987.80 19.50 ## 183 183 749 35200 1100.55 26.00 ## 184 184 751 32114 1474.48 26.14 ## 185 185 753 40209 1327.50 47.09 ## 186 186 769 19100 947.25 26.00 ## 187 187 770 41208 1266.14 29.08 ## 188 188 777 34275 1106.25 44.75 ## 189 189 780 36150 1660.56 29.50 ## 190 190 786 39186 1084.59 24.86 ## 191 191 798 29211 1555.85 30.11 ## 192 192 798 34171 1127.36 21.71 ## 193 193 798 34152 1034.02 31.52 ## 194 194 811 35126 1180.14 23.26 ## 195 195 814 30168 1610.70 39.68 ## 196 196 821 21127 2317.21 46.27 ## 197 197 863 34168 1481.20 30.68 ## 198 198 868 32109 1817.53 40.09 ## 199 199 887 34127 1593.72 31.27 ## 200 200 892 32143 1731.45 26.43 ## Stress_Category Physical_Demand Hiring_Potential ## 1 1 5.00 27.40 ## 2 1 3.97 19.78 ## 3 1 3.97 17.04 ## 4 1 3.95 11.08 ## 5 1 5.08 15.53 ## 6 1 6.98 12.10 ## 7 1 4.98 11.78 ## 8 1 5.09 11.08 ## 9 0 7.43 21.44 ## 10 1 7.00 33.07 ## 11 1 5.09 19.63 ## 12 1 4.23 16.74 ## 13 1 5.79 23.53 ## 14 1 7.98 11.96 ## 15 1 4.00 3.64 ## 16 1 6.04 8.56 ## 17 1 8.43 24.22 ## 18 1 6.47 17.55 ## 19 2 6.21 8.30 ## 20 1 4.09 -0.60 ## 21 1 9.43 17.43 ## 22 2 7.98 14.33 ## 23 1 4.76 5.39 ## 24 1 8.21 10.47 ## 25 1 6.36 4.27 ## 26 1 5.85 4.70 ## 27 1 9.79 19.93 ## 28 1 5.84 -7.24 ## 29 1 7.56 6.48 ## 30 1 7.26 10.00 ## 31 1 7.03 17.56 ## 32 1 12.79 21.58 ## 33 2 10.21 22.39 ## 34 1 6.06 24.10 ## 35 0 7.26 14.29 ## 36 1 8.86 14.70 ## 37 2 8.21 6.22 ## 38 1 7.23 6.65 ## 39 1 5.18 4.38 ## 40 0 7.26 6.48 ## 41 2 5.62 7.19 ## 42 1 7.98 12.76 ## 43 1 7.98 12.14 ## 44 2 10.00 8.86 ## 45 1 15.43 28.04 ## 46 1 8.56 9.71 ## 47 2 7.16 6.52 ## 48 1 7.98 5.81 ## 49 3 6.21 5.09 ## 50 1 7.12 -5.25 ## 51 2 6.85 1.00 ## 52 2 8.96 17.65 ## 53 2 9.56 19.11 ## 54 2 5.09 -1.61 ## 55 2 5.84 15.11 ## 56 2 12.98 14.74 ## 57 1 10.79 9.49 ## 58 3 4.09 21.29 ## 59 0 6.04 -8.00 ## 60 2 9.09 25.75 ## 61 1 6.92 -3.59 ## 62 2 10.28 1.36 ## 63 1 8.26 8.43 ## 64 2 9.09 25.90 ## 65 1 17.79 32.98 ## 66 1 5.03 -16.96 ## 67 1 9.30 15.69 ## 68 2 7.44 14.26 ## 69 1 9.98 -2.05 ## 70 0 9.39 -2.89 ## 71 1 14.19 5.66 ## 72 1 15.98 10.14 ## 73 2 5.47 7.54 ## 74 1 8.85 1.19 ## 75 2 9.96 14.44 ## 76 0 9.22 7.69 ## 77 1 14.98 6.45 ## 78 2 11.11 -2.65 ## 79 2 7.09 6.79 ## 80 2 4.85 -3.54 ## 81 2 6.85 8.79 ## 82 3 6.09 10.11 ## 83 2 10.90 20.65 ## 84 2 7.98 -2.58 ## 85 3 9.36 37.05 ## 86 1 6.03 -0.17 ## 87 2 9.85 6.35 ## 88 1 13.43 12.85 ## 89 0 8.98 -24.06 ## 90 0 7.98 -20.18 ## 91 1 7.41 -0.81 ## 92 2 11.90 9.42 ## 93 2 20.05 17.54 ## 94 3 13.15 20.14 ## 95 1 18.18 15.48 ## 96 2 6.97 1.07 ## 97 1 15.39 -3.66 ## 98 1 16.39 -0.69 ## 99 1 8.79 -9.28 ## 100 2 11.87 8.32 ## 101 3 15.90 20.58 ## 102 1 13.39 -3.68 ## 103 3 6.05 9.42 ## 104 2 6.00 4.18 ## 105 1 7.00 -9.33 ## 106 4 4.62 -0.95 ## 107 1 14.39 -3.53 ## 108 3 6.92 5.93 ## 109 0 12.13 -14.31 ## 110 1 12.39 -11.36 ## 111 3 7.25 4.70 ## 112 1 14.09 -7.86 ## 113 4 7.24 2.60 ## 114 4 6.00 -2.59 ## 115 3 20.72 10.10 ## 116 1 13.39 -3.71 ## 117 1 14.41 -6.53 ## 118 2 14.39 -2.68 ## 119 1 13.98 0.50 ## 120 1 10.82 -12.35 ## 121 5 10.87 5.53 ## 122 1 14.39 -10.48 ## 123 1 12.41 -11.43 ## 124 2 15.15 11.03 ## 125 3 8.58 16.57 ## 126 1 9.00 -8.37 ## 127 2 15.00 18.90 ## 128 4 6.85 -0.44 ## 129 2 24.11 13.97 ## 130 1 9.67 -17.68 ## 131 0 13.29 -10.53 ## 132 1 16.87 -12.78 ## 133 1 12.55 3.35 ## 134 0 13.53 -14.37 ## 135 0 13.53 -14.50 ## 136 3 14.77 5.45 ## 137 1 14.86 -6.78 ## 138 1 17.46 -12.62 ## 139 1 11.00 -5.00 ## 140 1 13.00 -2.93 ## 141 1 16.16 -6.75 ## 142 2 21.80 3.82 ## 143 3 9.85 -7.09 ## 144 2 10.85 2.65 ## 145 1 9.53 -31.88 ## 146 2 17.00 10.89 ## 147 2 18.47 10.38 ## 148 2 22.01 -1.24 ## 149 4 10.03 2.84 ## 150 2 18.85 6.86 ## 151 3 20.13 13.63 ## 152 1 17.57 -11.22 ## 153 3 9.58 1.08 ## 154 1 17.10 -40.76 ## 155 2 18.92 1.18 ## 156 3 8.23 -3.93 ## 157 2 13.46 5.19 ## 158 2 34.53 5.22 ## 159 2 11.00 0.53 ## 160 3 13.41 9.50 ## 161 1 27.85 0.79 ## 162 1 17.85 -13.62 ## 163 4 8.85 -0.84 ## 164 2 26.18 3.39 ## 165 2 12.03 0.67 ## 166 4 17.63 7.71 ## 167 1 22.00 -2.47 ## 168 2 32.85 -1.25 ## 169 1 17.00 -5.07 ## 170 3 9.58 -2.91 ## 171 2 21.68 2.28 ## 172 4 6.00 -8.53 ## 173 2 28.09 0.69 ## 174 2 26.86 -3.58 ## 175 2 25.46 -0.16 ## 176 5 43.23 18.22 ## 177 2 7.85 -11.25 ## 178 4 22.63 7.85 ## 179 2 36.55 8.74 ## 180 3 11.85 -4.17 ## 181 2 25.85 -6.40 ## 182 1 19.98 -5.50 ## 183 2 21.57 -2.00 ## 184 2 30.53 8.14 ## 185 4 15.85 2.09 ## 186 2 19.00 0.00 ## 187 2 30.73 -0.92 ## 188 4 9.85 -14.25 ## 189 2 30.77 4.50 ## 190 2 28.03 -14.14 ## 191 3 36.41 7.11 ## 192 2 25.67 -30.29 ## 193 3 26.00 -3.48 ## 194 2 29.08 -15.74 ## 195 3 21.26 4.68 ## 196 4 14.46 5.27 ## 197 3 33.46 -9.32 ## 198 4 38.87 0.09 ## 199 3 36.85 -12.73 ## 200 2 36.89 -19.57 ## Description ## 1 Researches_designs_develops_and_maintains_software_systems_along_with_hardware_development_for_medical_scientific_and_industrial_purposes ## 2 Applies_mathematical_theories_and_formulas_to_teach_or_solve_problems_in_a_business_educational_or_industrial_climate ## 3 Interprets_statistics_to_determine_probabilities_of_accidents_sickness_and_death_and_loss_of_property_from_theft_and_natural_disasters ## 4 Tabulates_analyzes_and_interprets_the_numeric_results_of_experiments_and_surveys ## 5 Plans_and_develops_computer_systems_for_businesses_and_scientific_institutions ## 6 Studies_the_physical_characteristics_motions_and_processes_of_the_earth's_atmosphere ## 7 Studies_the_relationship_of_plants_and_animals_to_their_environment ## 8 Analyzes_and_records_historical_information_from_a_specific_era_or_according_to_a_particular_area_of_expertise ## 9 Diagnoses_and_treats_hearing_problems_by_attempting_to_discover_the_range_nature_and_degree_of_hearing_function ## 10 Assists_dentists_in_diagnostic_and_therapeutic_aspects_of_a_group_or_private_dental_practice ## 11 Studies_human_behavior_by_examining_the_interaction_of_social_groups_and_institutions ## 12 Prepares_and_analyzes_financial_reports_to_assist_managers_in_business_industry_and_government ## 13 Assists_attorneys_in_preparation_of_legal_documents;_collection_of_depositions_and_affidavits;_and_investigation_research_and_analysis_of_legal_issues ## 14 Researches_and_develops_theories_concerning_the_physical_forces_of_nature ## 15 Related_to_careers_in_portfolio_management_the_financial_planner_offers_a_broad_range_of_services_aimed_at_assisting_individuals_in_managing_and_planning_their_financial_future ## 16 Studies_questions_concerning_the_nature_of_intellectual_concepts_and_attempts_to_construct_rational_theories_concerning_our_understanding_of_the_world_around_us ## 17 Develops_individualized_programs_of_activity_for_mentally_physically_developmentally_and_emotionally_impaired_persons_to_aid_them_in_in_achieving_self-reliance ## 18 Monitors_counsels_and_reports_on_the_progress_of_individuals_who_have_been_released_from_correctional_institutions_to_serve_parole ## 19 Designs_develops_and_tests_new_technologies_concerned_with_the_manufacture_of_commercial_and_military_aircraft_and_spacecraft ## 20 Studies_and_analyzes_the_effects_of_resources_such_as_land_labor_and_raw_materials_on_costs_and_their_relation_to_industry_and_government ## 21 Assesses_hearing_speech_and_language_disabilities_and_provides_treatment._Assists_individuals_with_communication_disorders_through_diagnostic_techniques ## 22 Uses_principles_of_physics_and_mathematics_to_understand_the_workings_of_the_universe ## 23 Loan_officers_help_people_apply_for_loans._This_lets_people_do_things_like_buy_a_house_or_a_car_or_pay_for_college ## 24 Plans_drilling_locations_and_effective_production_methods_for_optimal_access_to_oil_and_natural_gas ## 25 Assesses_patients'_dietary_needs_plans_menus_and_instructs_patients_and_their_families_about_proper_nutritional_care ## 26 Transforms_scientific_and_technical_information_into_readily_understandable_language ## 27 Diagnoses_visual_disorders_and_prescribes_and_administers_corrective_and_rehabilitative_treatments ## 28 Organizes_and_lists_the_instructions_for_computers_to_process_data_and_solve_problems_in_logical_order ## 29 Selects_and_organizes_materials_to_make_information_available_to_the_public ## 30 Performs_lab_analysis_for_diagnosis_and_treatment_of_disease ## 31 Transcribes_testimony_judicial_decisions_and_other_proceedings_of_a_court_of_law ## 32 Treats_physical_problems_by_manipulating_various_parts_of_the_body_especially_the_spinal_column ## 33 Plans_and_supervises_the_building_of_roads_bridges_tunnels_and_buildings ## 34 Transcribes_dictations_prepares_correspondence_and_assists_physicians_and_other_medical_scientists_in_compiling_reports_articles_speeches_and_conference_proceedings ## 35 Conducts_routine_laboratory_tests_and_analysis_used_in_the_detection_diagnosis_and_treatment_of_disease ## 36 Advises_physicians_and_patients_on_the_affects_of_drugs_and_medications;_prepares_and_dispenses_prescriptions ## 37 Plans_for_optimum_use_of_facilities_and_personnel_to_improve_industrial_efficiency ## 38 Assesses_an_establishment's_needs_and_available_products_and_buys_raw_materials_equipment_and_machinery_furniture_and_other_supplies_accordingly ## 39 Maintains_financial_records_and_prepares_statements_of_a_company's_income_and_daily_operating_expenses ## 40 Maintains_complete_accurate_and_up-to-date_medical_records_for_use_in_treatment_billing_and_statistical_surveys ## 41 Supervises_the_educational_curricula_and_day-to-day_activities_in_elementary_and_secondary_schools_as_well_as_colleges_and_universities ## 42 Makes_and_repairs_dentures_crowns_and_orthodontic_devices ## 43 Determines_tax_liability_and_collects_taxes_from_individuals_or_businesses ## 44 Creating_and_maintaining_layout_navigation_and_interactivity_of_intranet_and_internet_websites ## 45 Plans_and_directs_treatment_to_improve_mobility_and_alleviate_pain_in_persons_disabled_by_injury_or_disease ## 46 Helps_students_with_educational_and_social_problems_and_provides_vocational_guidance ## 47 Supervises_the_filming_and_editing_of_motion_pictures_for_entertainment_business_and_educational_purposes ## 48 Studies_the_life_functions_of_plants_and_animals_both_in_their_natural_habitats_and_under_experimental_or_abnormal_conditions ## 49 Conducts_research_designs_and_monitors_the_operation_and_maintenance_of_nuclear_reactors_and_power_plant_equipment ## 50 Assesses_and_analyzes_the_risks_inherent_in_insuring_potential_policy_holders_before_making_recommendations_to_the_insurance_companies_that_employ_them ## 51 Designs_and_develops_manufactured_products ## 52 Diagnoses_and_corrects_deviations_in_dental_growth_development_and_position ## 53 Plans_and_directs_activities_of_workers_and_arranges_for_the_exhibition_of_articles_of_interest_to_museum_visitors ## 54 Arbitrates_legal_matters_coming_under_the_jurisdiction_of_the_federal_government_using_a_thorough_knowledge_of_federal_statutes_and_legal_precedent ## 55 Interviews_prospective_employees_administers_tests_and_provides_information_about_company_policies_and_available_jobs ## 56 Studies_and_analyzes_the_physical_properties_of_the_earth's_surface_adding_to_our_knowledge_of_oil_and_gas_exploration_techniques ## 57 Diagnoses_and_treats_problems_of_the_feet_through_corrective_devices_medication_therapy_and_surgery ## 58 Collects_and_evaluates_data_to_make_recommendations_to_businesses_concerning_trends_in_consumer_purchasing ## 59 Manufactures_and_repairs_rings_bracelets_pins_and_necklaces_using_precious_or_semi-precious_metals_and_stones ## 60 Studies_the_social_customs_language_and_physical_attributes_of_people_throughout_the_world ## 61 Prepares_drawings_according_to_the_specifications_of_scientists_architects_and_engineers ## 62 Develops_mechanical_products_and_coordinates_the_operation_and_repair_of_power-using_and_power-producing_machinery ## 63 Fills_lens_prescriptions_and_fits_eyeglasses_and_contact_lenses ## 64 Studies_analyzes_and_collects_data_based_on_research_of_ancient_often_preliterate_cultures ## 65 Ministers_to_the_care_of_animals_through_the_use_of_preventative_and_diagnostic_techniques ## 66 Prepares_and_edits_typed_or_electronic_copies_of_handwritten_printed_or_magnetically_recorded_documents ## 67 Creates_hair_styles_and_advises_clients_about_caring_for_their_hair_between_appointments ## 68 Leads_a_congregation_in_worship_and_other_spiritual_services_provides_moral_guidance_to_members_and_participates_in_community_outreach ## 69 Develops_substances_through_research_of_properties_and_composition;_assists_industry_and_individuals_by_study_of_chemical_structures_of_products ## 70 Maintains_and_repairs_band_and_orchestral_instruments_of_all_kinds ## 71 Studies_the_behavior_life_processes_origins_and_diseases_of_animals ## 72 Conducts_research_into_range_problems_and_manages_range_lands_to_make_efficient_use_of_livestock_and_wildlife_without_destroying_their_habitat ## 73 Assists_individuals_families_and_groups_in_need_of_counseling_and_special_social_services ## 74 Plans_and_assembles_theatrical_sets_for_film_television_and_stage_productions ## 75 Examines_cleans_and_repairs_teeth_and_diagnoses_and_treats_diseases_and_abnormalities_of_the_mouth ## 76 Shampoos_trims_cuts_and_styles_hair_according_to_the_desires_of_customers ## 77 Examines_places_of_business_to_ensure_that_equipment_and_work_conditions_do_not_endanger_the_health_and_safety_of_employees ## 78 Conducts_research_and_plans_and_directs_design_testing_and_manufacture_of_electrical_equipment ## 79 Studies_human_behavior_emotion_and_mental_processes_and_provides_counseling_and_therapy_for_individuals ## 80 Plans_and_directs_the_editorial_activities_of_various_publications ## 81 Creates_fiction_and_non-fiction_books_either_on_assignment_from_editors_or_independently ## 82 Counsels_clients_in_legal_matters;_using_interpretation_of_laws_and_rulings_to_advise_and_represent_businesses_and_individuals ## 83 Performs_examinations_diagnoses_medical_conditions_and_prescribes_treatment_for_individuals_suffering_from_injury_discomfort_or_disease ## 84 Assists_engineers_in_planning_design_and_development_by_applying_a_practical_knowledge_of_civil_mechanical_or_industrial_engineering ## 85 Aids_in_the_performance_of_essential_procedures_which_free_physicians_to_attend_to_more_specialized_aspects_of_their_work ## 86 Cashes_checks_makes_deposits_and_withdrawals_and_handles_a_variety_of_other_transactions_for_bank_customers ## 87 Creates_artwork_independently_usually_in_the_form_of_painting_drawing_sculpture_or_other_visual_mediums ## 88 Treats_and_rehabilitates_patients_suffering_from_cardiopulmonary_(heart_and_lung)_ailments_which_interfere_with_normal_breathing ## 89 Participates_in_the_assembly_of_books_or_magazines_for_commercial_printing_companies ## 90 Develops_photographic_film_makes_prints_and_slides_prepares_enlargements_and_retouches_photographs ## 91 Supervises_elementary_school_students_in_the_classroom_school_yard_and_in_the_cafeteria_operates_audio-visual_equipment_records_grades_and_prepares_educational_materials ## 92 Studies_diagnoses_and_treats_mental_emotional_and_behavioral_disorders ## 93 Installs_and_services_air-conditioning_and_furnace_systems_in_businesses_and_residences ## 94 Assists_physicians_in_administering_holistic_medical_care_and_treatment_to_assigned_patients_in_clinics_hospitals_public_health_centers_and_health_maintenance_organizations ## 95 Monitors_and_controls_processes_and_equipment_to_remove_harmful_waste_materials_from_sewage ## 96 Oversees_the_successful_and_profitable_operation_of_hotels_and_motels ## 97 Installs_services_cleans_tests_and_repairs_telephones_and_switchboard_systems ## 98 Maintains_inspects_and_repairs_industrial_machinery ## 99 Runs_and_maintains_the_equipment_used_to_transmit_radio_and_television_messages ## 100 Introduces_children_to_the_basics_of_mathematics_language_science_and_social_studies_and_assists_other_aspects_of_child_development ## 101 Diagnoses_ailments_and_performs_operations_to_repair_reconstruct_remove_or_replace_organs_limbs_and_bodily_systems ## 102 Installs_and_repairs_electronic_equipment_for_military_installations_manufacturers_and_businesses ## 103 Sells_insurance_and_advises_clients_about_amount_and_type_of_coverage_based_on_needs_and_circumstances ## 104 Greets_visitors_to_offices_answers_questions_and_refers_customers_to_appropriate_staff ## 105 Receives_payments_makes_change_and_provides_receipts_for_goods_sold ## 106 Negotiates_to_procure_accounts_and_supervises_advertising_campaigns_for_products_companies_and_organizations ## 107 Performs_maintenance_and_repairs_on_coin-operated_vending_and_amusement_machines ## 108 Plans_and_designs_spaces_to_be_constructed_or_remodeled_according_to_specifications_of_clients ## 109 Develops_assembles_and_tests_electrical_equipment_according_to_principles_of_electrical_engineering ## 110 Repairs_malfunctions_maintains_service_according_to_manufacturers'_schedules_and_sometimes_installs_computers_and_peripheral_equipment ## 111 Facilitates_the_purchase_and_sale_of_stocks_bonds_and_other_securities_for_individual_and_institutional_clients ## 112 Operates_industrial_trucks_and_tractors_to_move_products_and_raw_materials_for_manufacturing_firms ## 113 Helps_governmental_bodies_businesses_and_individuals_maintain_a_positive_image_with_the_public ## 114 Formulates_the_policies_and_directs_the_operations_of_private_and_publicly-held_companies ## 115 Supervises_the_work_of_employees_and_ensures_that_equipment_and_materials_are_being_used_properly_and_effectively ## 116 Adjusts_piano_strings_to_achieve_proper_musical_pitch ## 117 Builds_new_furniture_and_restores_worn_furniture_using_a_thorough_knowledge_of_fabrics_and_manufacturing_techniques ## 118 Installs_repairs_and_maintains_residential_and_commercial_phone_systems ## 119 Researches_methods_to_improve_quantity_and_quality_of_yields_from_farm_crops_and_livestock_and_attempts_to_find_practical_solutions_to_problems_in_agriculture ## 120 Typesetters_operate_keyboards_to_prepare_print_documents_while_compositors_lay_out_pages_check_proofs_and_make_corrections ## 121 Operates_an_aircraft_to_transport_passengers_and_cargo_to_appointed_destinations ## 122 Performs_major_and_routine_maintenance_on_a_variety_of_electrical_home_appliances ## 123 Repairs_and_maintains_business_machines ## 124 Prepares_bodies_for_burial_and_arranges_and_directs_funerals ## 125 Acts_as_intermediary_between_buyer_and_seller_in_real_estate_transactions_usually_by_being_the_prime_salesperson_of_a_property ## 126 Operates_manual_or_computerized_telephone_switchboards_and_assists_in_the_placement_of_local_and_long_distance_calls ## 127 Assists_doctors_and_registered_nurses_in_the_care_of_physically_or_mentally_ill_patients ## 128 Prepares_and_delivers_news_and_related_presentations_over_the_air_on_radio_and_television ## 129 Establishes_official_land_and_aquatic_boundaries_measures_construction_and_mining_sites_writes_technical_property_descriptions_for_deeds_and_leases_and_prepares_maps_and_charts ## 130 Monitors_the_movement_of_shipments_of_merchandise_into_and_out_of_places_of_business ## 131 Follows_design_instructions_operates_a_sewing_machine_to_join_reinforce_and_decorate_parts_of_garments_or_other_textiles ## 132 Constructs_and_repairs_precision_tools_gauges_holding_devices_and_stamping_tools_for_the_machine_industry ## 133 Protects_property_from_damages_incurred_by_theft_fire_and_vandalism ## 134 Assembles_parts_or_contributes_to_the_final_assembly_of_automobiles_and_trucks ## 135 Works_on_sub-assembly_or_final_assembly_of_products_such_as_machinery_electronic_equipment_or_aircraft ## 136 Operates_an_electric_diesel-electric_or_gas-turbine-_electric_locomotive_to_transport_passengers_and_freight ## 137 Operates_computerized_machines_in_the_manufacture_of_industrial_parts ## 138 Operates_machinery_in_manufacturing_plants_for_fabrication_of_industrial_parts ## 139 Takes_customer_orders_serves_food_and_drink_and_prepares_meal_checks ## 140 Mixes_and_serves_drinks_to_customers_of_a_tavern_restaurant_or_lounge ## 141 Cleans_offices_and_other_spaces_within_buildings_and_keeps_areas_in_good_condition ## 142 Builds_and_repairs_water_waste_disposal_drainage_and_gas_delivery_systems_for_residential_commercial_and_industrial_structures ## 143 Designs_articles_or_complete_lines_of_clothing_for_men_women_or_children ## 144 Uses_shutter-operated_cameras_and_photographic_emulsions_to_visually_portray_a_variety_of_subjects ## 145 Refurbishes_and_mends_worn_shoes_saddles_and_boots ## 146 Assists_practical_nurses_in_caring_for_patients._Nursing_homes_and_hospitals_are_the_primary_employers_of_nurse's_aides ## 147 Organizes_and_supervises_a_variety_of_leisure_activities_including_sports_arts_crafts_drama_singing_dancing_and_story_telling ## 148 Maps_layout_and_installs_and_repairs_electrical_wiring_and_fixtures ## 149 Tends_to_the_care_and_comfort_of_passengers_on_commercial_and_corporate_aircraft ## 150 Measures_cuts_and_installs_glass_in_residential_and_commercial_buildings ## 151 Cleanses_nuclear_power_plant_equipment_and_personnel_of_irradiated_material ## 152 Mends_cosmetic_and_structural_damage_to_car_truck_and_bus_chassis ## 153 Represents_wholesalers_who_distribute_products_to_stores_manufacturers_and_businesses ## 154 Operates_drilling_machines_to_place_holes_in_metal_or_nonmetal_pieces_according_to_predetermined_specifications ## 155 Performs_scheduled_inspections_maintenance_and_repairs_on_commercial_and_private_aircraft ## 156 Orders_merchandise_and_maintains_inventory_control_for_wholesale_and_retail_sales_firms ## 157 Drives_limousines_or_other_vehicles_to_transport_passengers_to_and_from_specified_locations ## 158 Manages_the_successful_operation_of_a_crop_livestock_dairy_or_poultry_farm ## 159 Provides_courteous_and_efficient_service_to_customers_in_retail_stores ## 160 Supervises_inmates'_activities_and_enforces_regulations_in_jails_prisons_and_other_correctional_facilities ## 161 Installs_and_prepares_surfaces_of_drywall_panels_for_commercial_or_residential_interiors ## 162 Lays_tile_or_carpets_in_homes_and_businesses ## 163 Entertains_informs_and_instructs_audiences_by_interpreting_dramatic_roles_on_stage_film_television_or_radio ## 164 Operates_monitors_and_repairs_power_plants_and_industrial_heating_cooling_and_ventilation_systems ## 165 Transports_passengers_according_to_a_specific_schedule_along_metropolitan_and_community_routes ## 166 Patrols_roads_and_highways_and_enforces_traffic_regulations_and_criminal_statutes ## 167 Cleans_the_plates_glasses_and_silverware_used_by_patrons_of_an_eating_establishment_and_the_pots_pans_and_cooking_utensils_used_by_chefs ## 168 Erects_structures_such_as_fireplaces_and_walls_with_brick_and_other_masonry_materials ## 169 May_perform_a_variety_of_services_including_cleaning_and_upkeep_for_residential_and_institutional_employers ## 170 Negotiates_contracts_with_clients_for_advertising_in_publications_and_on_radio_and_television ## 171 Operates_16-wheeled_tractor-trailer_to_transport_durable_and_perishable_goods_from_producers_to_consumers ## 172 Makes_arrangements_for_travel_according_to_the_specific_needs_of_individuals_and_businesses ## 173 Operates_one_or_more_machines_used_in_extractive_or_construction_work ## 174 Delivers_and_collects_mail_along_prearranged_rural_and_urban_routes ## 175 Assists_in_the_construction_repairing_and_remodeling_of_buildings_and_other_structures ## 176 Protects_individuals_and_saves_lives_and_property_from_the_ravages_of_fire ## 177 Selects_and_plays_records_or_tapes;_comments_on_areas_of_interest_to_a_particular_radio_audience ## 178 Provides_protection_against_crime_investigates_criminal_activity_and_works_with_the_public_on_crime-prevention_measures ## 179 Collects_refuse_on_a_designated_municipal_route_and_transports_trash_to_disposal_plants_or_landfill_areas ## 180 Instructs_performers_and_develops_and_interprets_routines_for_stage_and_other_presentations ## 181 Applies_the_plaster_finish_to_interior_walls_and_ceilings_and_cement_finish_and_stucco_to_exterior_surfaces ## 182 Prepares_meat_for_sale_to_distributors_supermarket_customers_and_other_consumers ## 183 Diagnoses_problems_with_automotive_vehicles_makes_repairs_and_performs_routine_maintenance ## 184 Directs_and_takes_part_in_activities_involved_in_the_raising_of_cattle_for_milk_production ## 185 Photographs_newsworthy_events_for_publication_in_newspapers_and_magazines ## 186 Cares_for_infants_and_toddlers_when_parents_are_at_work_or_are_unable_to_do_so_for_other_reasons ## 187 Constructs_installs_and_maintains_sheet_metal_products_for_home_commercial_and_industrial_use ## 188 Covers_newsworthy_events_for_newspapers_magazines_and_television_news_programs ## 189 Performs_any_number_of_tasks_involved_in_the_operation_of_ships_boats_barges_or_dredges ## 190 Loads_and_unloads_cargo_from_vessels_routes_cargo_to_proper_locations ## 191 Assists_construction_trade_workers_by_performing_a_wide_variety_of_tasks_requiring_physical_labor ## 192 Monitors_public_utility_meters_and_records_volume_of_consumption_by_customers ## 193 Prepares_surfaces_and_applies_paints_varnishes_and_finishes_to_the_interiors_and_exteriors_of_houses_and_other_structures ## 194 Joins_or_repairs_metal_surfaces_through_the_application_of_heat ## 195 Attends_to_situations_which_demand_immediate_medical_attention_such_as_automobile_accidents_heart_attacks_and_gunshot_wounds ## 196 Operates_a_taxi_cab_over_the_streets_and_roads_of_a_municipality_picking_up_and_dropping_off_passengers_by_request ## 197 Installs_roofs_on_new_buildings_performs_repairs_on_old_roofs_and_re-roofs_old_buildings ## 198 Fells_cuts_and_transports_timber_to_be_processed_into_lumber_paper_and_other_wood_products ## 199 Raises_the_steel_framework_of_buildings_bridges_and_other_structures ## 200 Performs_routine_physical_labor_and_maintenance_on_oil_rigs_and_pipelines_both_on_and_off_shore ## cleandesc ## 1 researches designs develops maintains software systems along hardware development medical scientific industrial purposes ## 2 applies mathematical theories formulas teach solve problems business educational industrial climate ## 3 interprets statistics determine probabilities accidents sickness death loss property theft natural disasters ## 4 tabulates analyzes interprets numeric results experiments surveys ## 5 plans develops computer systems businesses scientific institutions ## 6 studies physical characteristics motions processes earth's atmosphere ## 7 studies relationship plants animals environment ## 8 analyzes records historical information specific era according particular area expertise ## 9 diagnoses treats hearing problems attempting discover range nature degree hearing function ## 10 assists dentists diagnostic therapeutic aspects group private dental practice ## 11 studies human behavior examining interaction social groups institutions ## 12 prepares analyzes financial reports assist managers business industry government ## 13 assists attorneys preparation legal documents; collection depositions affidavits; investigation research analysis legal issues ## 14 researches develops theories concerning physical forces nature ## 15 related careers portfolio management financial planner offers broad range services aimed assisting individuals managing planning financial future ## 16 studies questions concerning nature intellectual concepts attempts construct rational theories concerning understanding world around us ## 17 develops individualized programs activity mentally physically developmentally emotionally impaired persons aid achieving self-reliance ## 18 monitors counsels reports progress individuals released correctional institutions serve parole ## 19 designs develops tests new technologies concerned manufacture commercial military aircraft spacecraft ## 20 studies analyzes effects resources land labor raw materials costs relation industry government ## 21 assesses hearing speech language disabilities provides treatment. assists individuals communication disorders diagnostic techniques ## 22 uses principles physics mathematics understand workings universe ## 23 loan officers help people apply loans. lets people things like buy house car pay college ## 24 plans drilling locations effective production methods optimal access oil natural gas ## 25 assesses patients' dietary needs plans menus instructs patients families proper nutritional care ## 26 transforms scientific technical information readily understandable language ## 27 diagnoses visual disorders prescribes administers corrective rehabilitative treatments ## 28 organizes lists instructions computers process data solve problems logical order ## 29 selects organizes materials make information available public ## 30 performs lab analysis diagnosis treatment disease ## 31 transcribes testimony judicial decisions proceedings court law ## 32 treats physical problems manipulating various parts body especially spinal column ## 33 plans supervises building roads bridges tunnels buildings ## 34 transcribes dictations prepares correspondence assists physicians medical scientists compiling reports articles speeches conference proceedings ## 35 conducts routine laboratory tests analysis used detection diagnosis treatment disease ## 36 advises physicians patients affects drugs medications; prepares dispenses prescriptions ## 37 plans optimum use facilities personnel improve industrial efficiency ## 38 assesses establishment's needs available products buys raw materials equipment machinery furniture supplies accordingly ## 39 maintains financial records prepares statements company's income daily operating expenses ## 40 maintains complete accurate --date medical records use treatment billing statistical surveys ## 41 supervises educational curricula day--day activities elementary secondary schools well colleges universities ## 42 makes repairs dentures crowns orthodontic devices ## 43 determines tax liability collects taxes individuals businesses ## 44 creating maintaining layout navigation interactivity intranet internet websites ## 45 plans directs treatment improve mobility alleviate pain persons disabled injury disease ## 46 helps students educational social problems provides vocational guidance ## 47 supervises filming editing motion pictures entertainment business educational purposes ## 48 studies life functions plants animals natural habitats experimental abnormal conditions ## 49 conducts research designs monitors operation maintenance nuclear reactors power plant equipment ## 50 assesses analyzes risks inherent insuring potential policy holders making recommendations insurance companies employ ## 51 designs develops manufactured products ## 52 diagnoses corrects deviations dental growth development position ## 53 plans directs activities workers arranges exhibition articles interest museum visitors ## 54 arbitrates legal matters coming jurisdiction federal government using thorough knowledge federal statutes legal precedent ## 55 interviews prospective employees administers tests provides information company policies available jobs ## 56 studies analyzes physical properties earth's surface adding knowledge oil gas exploration techniques ## 57 diagnoses treats problems feet corrective devices medication therapy surgery ## 58 collects evaluates data make recommendations businesses concerning trends consumer purchasing ## 59 manufactures repairs rings bracelets pins necklaces using precious semi-precious metals stones ## 60 studies social customs language physical attributes people throughout world ## 61 prepares drawings according specifications scientists architects engineers ## 62 develops mechanical products coordinates operation repair power-using power-producing machinery ## 63 fills lens prescriptions fits eyeglasses contact lenses ## 64 studies analyzes collects data based research ancient often preliterate cultures ## 65 ministers care animals use preventative diagnostic techniques ## 66 prepares edits typed electronic copies handwritten printed magnetically recorded documents ## 67 creates hair styles advises clients caring hair appointments ## 68 leads congregation worship spiritual services provides moral guidance members participates community outreach ## 69 develops substances research properties composition; assists industry individuals study chemical structures products ## 70 maintains repairs band orchestral instruments kinds ## 71 studies behavior life processes origins diseases animals ## 72 conducts research range problems manages range lands make efficient use livestock wildlife without destroying habitat ## 73 assists individuals families groups need counseling special social services ## 74 plans assembles theatrical sets film television stage productions ## 75 examines cleans repairs teeth diagnoses treats diseases abnormalities mouth ## 76 shampoos trims cuts styles hair according desires customers ## 77 examines places business ensure equipment work conditions endanger health safety employees ## 78 conducts research plans directs design testing manufacture electrical equipment ## 79 studies human behavior emotion mental processes provides counseling therapy individuals ## 80 plans directs editorial activities various publications ## 81 creates fiction non-fiction books either assignment editors independently ## 82 counsels clients legal matters; using interpretation laws rulings advise represent businesses individuals ## 83 performs examinations diagnoses medical conditions prescribes treatment individuals suffering injury discomfort disease ## 84 assists engineers planning design development applying practical knowledge civil mechanical industrial engineering ## 85 aids performance essential procedures free physicians attend specialized aspects work ## 86 cashes checks makes deposits withdrawals handles variety transactions bank customers ## 87 creates artwork independently usually form painting drawing sculpture visual mediums ## 88 treats rehabilitates patients suffering cardiopulmonary (heart lung) ailments interfere normal breathing ## 89 participates assembly books magazines commercial printing companies ## 90 develops photographic film makes prints slides prepares enlargements retouches photographs ## 91 supervises elementary school students classroom school yard cafeteria operates audio-visual equipment records grades prepares educational materials ## 92 studies diagnoses treats mental emotional behavioral disorders ## 93 installs services air-conditioning furnace systems businesses residences ## 94 assists physicians administering holistic medical care treatment assigned patients clinics hospitals public health centers health maintenance organizations ## 95 monitors controls processes equipment remove harmful waste materials sewage ## 96 oversees successful profitable operation hotels motels ## 97 installs services cleans tests repairs telephones switchboard systems ## 98 maintains inspects repairs industrial machinery ## 99 runs maintains equipment used transmit radio television messages ## 100 introduces children basics mathematics language science social studies assists aspects child development ## 101 diagnoses ailments performs operations repair reconstruct remove replace organs limbs bodily systems ## 102 installs repairs electronic equipment military installations manufacturers businesses ## 103 sells insurance advises clients amount type coverage based needs circumstances ## 104 greets visitors offices answers questions refers customers appropriate staff ## 105 receives payments makes change provides receipts goods sold ## 106 negotiates procure accounts supervises advertising campaigns products companies organizations ## 107 performs maintenance repairs coin-operated vending amusement machines ## 108 plans designs spaces constructed remodeled according specifications clients ## 109 develops assembles tests electrical equipment according principles electrical engineering ## 110 repairs malfunctions maintains service according manufacturers' schedules sometimes installs computers peripheral equipment ## 111 facilitates purchase sale stocks bonds securities individual institutional clients ## 112 operates industrial trucks tractors move products raw materials manufacturing firms ## 113 helps governmental bodies businesses individuals maintain positive image public ## 114 formulates policies directs operations private publicly-held companies ## 115 supervises work employees ensures equipment materials used properly effectively ## 116 adjusts piano strings achieve proper musical pitch ## 117 builds new furniture restores worn furniture using thorough knowledge fabrics manufacturing techniques ## 118 installs repairs maintains residential commercial phone systems ## 119 researches methods improve quantity quality yields farm crops livestock attempts find practical solutions problems agriculture ## 120 typesetters operate keyboards prepare print documents compositors lay pages check proofs make corrections ## 121 operates aircraft transport passengers cargo appointed destinations ## 122 performs major routine maintenance variety electrical home appliances ## 123 repairs maintains business machines ## 124 prepares bodies burial arranges directs funerals ## 125 acts intermediary buyer seller real estate transactions usually prime salesperson property ## 126 operates manual computerized telephone switchboards assists placement local long distance calls ## 127 assists doctors registered nurses care physically mentally ill patients ## 128 prepares delivers news related presentations air radio television ## 129 establishes official land aquatic boundaries measures construction mining sites writes technical property descriptions deeds leases prepares maps charts ## 130 monitors movement shipments merchandise places business ## 131 follows design instructions operates sewing machine join reinforce decorate parts garments textiles ## 132 constructs repairs precision tools gauges holding devices stamping tools machine industry ## 133 protects property damages incurred theft fire vandalism ## 134 assembles parts contributes final assembly automobiles trucks ## 135 works sub-assembly final assembly products machinery electronic equipment aircraft ## 136 operates electric diesel-electric gas-turbine- electric locomotive transport passengers freight ## 137 operates computerized machines manufacture industrial parts ## 138 operates machinery manufacturing plants fabrication industrial parts ## 139 takes customer orders serves food drink prepares meal checks ## 140 mixes serves drinks customers tavern restaurant lounge ## 141 cleans offices spaces within buildings keeps areas good condition ## 142 builds repairs water waste disposal drainage gas delivery systems residential commercial industrial structures ## 143 designs articles complete lines clothing men women children ## 144 uses shutter-operated cameras photographic emulsions visually portray variety subjects ## 145 refurbishes mends worn shoes saddles boots ## 146 assists practical nurses caring patients. nursing homes hospitals primary employers nurse's aides ## 147 organizes supervises variety leisure activities including sports arts crafts drama singing dancing story telling ## 148 maps layout installs repairs electrical wiring fixtures ## 149 tends care comfort passengers commercial corporate aircraft ## 150 measures cuts installs glass residential commercial buildings ## 151 cleanses nuclear power plant equipment personnel irradiated material ## 152 mends cosmetic structural damage car truck bus chassis ## 153 represents wholesalers distribute products stores manufacturers businesses ## 154 operates drilling machines place holes metal nonmetal pieces according predetermined specifications ## 155 performs scheduled inspections maintenance repairs commercial private aircraft ## 156 orders merchandise maintains inventory control wholesale retail sales firms ## 157 drives limousines vehicles transport passengers specified locations ## 158 manages successful operation crop livestock dairy poultry farm ## 159 provides courteous efficient service customers retail stores ## 160 supervises inmates' activities enforces regulations jails prisons correctional facilities ## 161 installs prepares surfaces drywall panels commercial residential interiors ## 162 lays tile carpets homes businesses ## 163 entertains informs instructs audiences interpreting dramatic roles stage film television radio ## 164 operates monitors repairs power plants industrial heating cooling ventilation systems ## 165 transports passengers according specific schedule along metropolitan community routes ## 166 patrols roads highways enforces traffic regulations criminal statutes ## 167 cleans plates glasses silverware used patrons eating establishment pots pans cooking utensils used chefs ## 168 erects structures fireplaces walls brick masonry materials ## 169 may perform variety services including cleaning upkeep residential institutional employers ## 170 negotiates contracts clients advertising publications radio television ## 171 operates 16-wheeled tractor-trailer transport durable perishable goods producers consumers ## 172 makes arrangements travel according specific needs individuals businesses ## 173 operates one machines used extractive construction work ## 174 delivers collects mail along prearranged rural urban routes ## 175 assists construction repairing remodeling buildings structures ## 176 protects individuals saves lives property ravages fire ## 177 selects plays records tapes; comments areas interest particular radio audience ## 178 provides protection crime investigates criminal activity works public crime-prevention measures ## 179 collects refuse designated municipal route transports trash disposal plants landfill areas ## 180 instructs performers develops interprets routines stage presentations ## 181 applies plaster finish interior walls ceilings cement finish stucco exterior surfaces ## 182 prepares meat sale distributors supermarket customers consumers ## 183 diagnoses problems automotive vehicles makes repairs performs routine maintenance ## 184 directs takes part activities involved raising cattle milk production ## 185 photographs newsworthy events publication newspapers magazines ## 186 cares infants toddlers parents work unable reasons ## 187 constructs installs maintains sheet metal products home commercial industrial use ## 188 covers newsworthy events newspapers magazines television news programs ## 189 performs number tasks involved operation ships boats barges dredges ## 190 loads unloads cargo vessels routes cargo proper locations ## 191 assists construction trade workers performing wide variety tasks requiring physical labor ## 192 monitors public utility meters records volume consumption customers ## 193 prepares surfaces applies paints varnishes finishes interiors exteriors houses structures ## 194 joins repairs metal surfaces application heat ## 195 attends situations demand immediate medical attention automobile accidents heart attacks gunshot wounds ## 196 operates taxi cab streets roads municipality picking dropping passengers request ## 197 installs roofs new buildings performs repairs old roofs re-roofs old buildings ## 198 fells cuts transports timber processed lumber paper wood products ## 199 raises steel framework buildings bridges structures ## 200 performs routine physical labor maintenance oil rigs pipelines shore ## jt ## 1 Software Engineer ## 2 Mathematician ## 3 Actuary ## 4 Statistician ## 5 Computer Systems Analyst ## 6 Meteorologist ## 7 Biologist ## 8 Historian ## 9 Audiologist ## 10 Dental Hygenist ## 11 Sociologist ## 12 Accountant ## 13 Paralegal Assistant ## 14 Physicist ## 15 Financial Planner ## 16 Philosopher ## 17 Occupational Therapist ## 18 Parole Officer ## 19 Aerospace Engineer ## 20 Economist ## 21 Speech Pathologist ## 22 Astronomer ## 23 Loan Officer ## 24 Petroleum Engineer ## 25 Dietitian ## 26 Technical Writer ## 27 Optometrist ## 28 Computer Programmer ## 29 Librarian ## 30 Medical Technologist ## 31 Stenographer/Court Reporter ## 32 Chiropractor ## 33 Civil Engineer ## 34 Medical Secretary ## 35 Medical Laboratory Technician ## 36 Pharmacist ## 37 Industrial Engineer ## 38 Purchasing Agent ## 39 Bookkeeper ## 40 Medical Records Technician ## 41 School Principal ## 42 Dental Laboratory Technician ## 43 Tax Examiner/Collector ## 44 Web Developer ## 45 Physical Therapist ## 46 Vocational Counselor ## 47 Motion Picture Editor ## 48 Physiologist ## 49 Nuclear Engineer ## 50 Insurance Underwriter ## 51 Industrial Designer ## 52 Orthodontist ## 53 Museum Curator ## 54 Judge ## 55 Personnel Recruiter ## 56 Geologist ## 57 Podiatrist ## 58 Market Research Analyst ## 59 Jeweler ## 60 Anthropologist ## 61 Architectural Drafter ## 62 Mechanical Engineer ## 63 Optician ## 64 Archeologist ## 65 Veterinarian ## 66 Typist/Word Processor ## 67 Cosmetologist ## 68 Clergy ## 69 Chemist ## 70 Musical Instrument Repairer ## 71 Zoologist ## 72 Conservationist ## 73 Social Worker ## 74 Set Designer ## 75 Dentist ## 76 Barber ## 77 Occupational Safety/Health Inspector ## 78 Electrical Engineer ## 79 Psychologist ## 80 Publication Editor ## 81 Author ## 82 Attorney ## 83 Physician (General Practice) ## 84 Engineering Technician ## 85 Physician Assistant ## 86 Bank Teller ## 87 Artist (Fine Art) ## 88 Respiratory Therapist ## 89 Bookbinder ## 90 Photographic Process Worker ## 91 Teacher's Aide ## 92 Psychiatrist ## 93 Heating/Refrigeration Mechanic ## 94 Nurse (Registered) ## 95 Sewage Plant Operator ## 96 Hotel Manager ## 97 Telephone Installer/Repairer ## 98 Industrial Machine Repairer20 ## 99 Broadcast Technician ## 100 Teacher ## 101 Surgeon ## 102 Electrical Equipment Repairer ## 103 Insurance Agent ## 104 Receptionist ## 105 Cashier ## 106 Advertising Account Executive ## 107 Vending Machine Repairer ## 108 Architect ## 109 Electrical Technician ## 110 Computer Service Technician ## 111 Stockbroker ## 112 Forklift Operator ## 113 Public Relations Executive ## 114 Corporate Executive (Senior) ## 115 Construction Foreman ## 116 Piano Tuner ## 117 Furniture Upholsterer ## 118 Communications Equipment Mechanic ## 119 Agricultural Scientist ## 120 Compositor/Typesetter ## 121 Commercial Airline Pilot ## 122 Appliance Repairer ## 123 Office Machine Repairer ## 124 Undertaker ## 125 Real Estate Agent ## 126 Telephone Operator ## 127 Nurse (Licensed Practical) ## 128 Newscaster ## 129 Surveyor ## 130 Shipping/Receiving Clerk ## 131 Dressmaker ## 132 Tool-And-Die Maker ## 133 Guard ## 134 Automobile Assembler ## 135 Precision Assembler ## 136 Railroad Conductor ## 137 Machine Tool Operator ## 138 Machinist ## 139 Waiter/Waitress ## 140 Bartender ## 141 Janitor ## 142 Plumber ## 143 Fashion Designer ## 144 Photographer ## 145 Shoe Maker/Repairer ## 146 Nurse's Aide ## 147 Recreation Worker ## 148 Electrician ## 149 Flight Attendant ## 150 Glazier ## 151 Nuclear Decontamination Technician ## 152 Automobile Body Repairer ## 153 Sales Representative (Wholesale) ## 154 Drill-Press Operator ## 155 Aircraft Mechanic ## 156 Buyer ## 157 Chauffeur ## 158 Farmer ## 159 Salesperson (Retail) ## 160 Corrections Officer ## 161 Drywall Applicator/Finisher ## 162 Carpet/Tile Installer ## 163 Actor ## 164 Stationary Engineer ## 165 Bus Driver ## 166 Highway Patrol Officer ## 167 Dishwasher ## 168 Bricklayer ## 169 Maid ## 170 Advertising Salesperson ## 171 Truck Driver ## 172 Travel Agent ## 173 Construction Machinery Operator ## 174 Mail Carrier ## 175 Carpenter ## 176 Firefighter ## 177 Disc Jockey ## 178 Police Officer ## 179 Garbage Collector ## 180 Choreographer ## 181 Plasterer ## 182 Butcher ## 183 Automobile Mechanic ## 184 Dairy Farmer ## 185 Photojournalist ## 186 Child Care Worker ## 187 Sheet Metal Worker ## 188 Reporter (Newspaper) ## 189 Sailor ## 190 Stevedore ## 191 Construction Worker ## 192 Meter Reader ## 193 Painter ## 194 Welder ## 195 Emergency Medical Technician ## 196 Taxi Driver ## 197 Roofer ## 198 Lumberjack ## 199 Ironworker ## 200 Roustaboutset.seed(1234) subset_int <- sample(nrow(js2),floor(nrow(js2)*0.8)) # 80% training + 20% testing js_train<-js2[subset_int, ] js_test<-js2[-subset_int, ] js_dtm_train<-js_dtm[subset_int, ] js_dtm_test<-js_dtm[-subset_int, ] corpus_train<-corpus_clean[subset_int] corpus_test<-corpus_clean[-subset_int] js_train$sl<-js_train$Stress_Category %in% c(0:2) js_train$sl<-factor(js_train$sl, levels=c(T, F), labels = c("Low Stress", "High Stress")) js_test$sl<-js_test$Stress_Category %in% c(0:2) js_test$sl<-factor(js_test$sl, levels=c(T, F), labels = c("Low Stress", "High Stress")) prop.table(table(js_train$sl))## ## Low Stress High Stress ## 0.8125 0.1875Generate a word cloud to visualize the job description text.
#install.packages("wordcloud", repos = "http://cran.us.r-project.org") library(wordcloud)wordcloud(corpus_clean, scale=c(2,.5),min.freq = 5, random.order = T, colors=brewer.pal(5, "Dark2"))
Graphically visualize the difference between low and high Stress_Category graph.
low<-subset(js_train, sl=="Low Stress") high<-subset(js_train, sl=="High Stress") wordcloud(low$jt, min.freq = 1,max.words = 80, scale=c(1.5, .5),colors=brewer.pal(3, "Dark2"))## Warning in tm_map.SimpleCorpus(corpus, tm::removePunctuation): transformation ## drops documents## Warning in tm_map.SimpleCorpus(corpus, function(x) tm::removeWords(x, ## tm::stopwords())): transformation drops documents
wordcloud(high$jt, min.freq = 1,max.words = 80, scale=c(1.5, .5),colors=brewer.pal(3, "Dark2"))## Warning in tm_map.SimpleCorpus(corpus, tm::removePunctuation): transformation ## drops documents ## Warning in tm_map.SimpleCorpus(corpus, tm::removePunctuation): transformation ## drops documents
### Low Stress involved industrial, engineer, worker, health and repairer. High Stress career involved agent, executive, officer, research, surgeon,construction and police. Technician is not significant in term of stress level.Transform the word count features into categorical data
summary(findFreqTerms(js_dtm_train, 3))## Length Class Mode ## 115 character characterjs_dict<-as.character(findFreqTerms(js_dtm_train, 3)) js_train2<-DocumentTermMatrix(corpus_train, list(dictionary=js_dict)) js_test2<-DocumentTermMatrix(corpus_test, list(dictionary=js_dict)) convert_counts <- function(wordFreq) { wordFreq <- ifelse(wordFreq > 0, 1, 0) wordFreq <- factor(wordFreq, levels = c(0, 1), labels = c("No", "Yes")) return(wordFreq) } js_train2 <- apply(js_train2, MARGIN = 2, convert_counts) js_test2 <- apply(js_test2, MARGIN = 2, convert_counts) dim(js_train2)## [1] 160 115table(js_train2)## js_train2 ## No Yes ## 17862 538Ignore those low frequency words and report the sparsity of your categorical data matrix with or without delete those low frequency words. Note that the
sparsityof a matrix is the fraction:\(Sparsity(A) =\frac{number \;of \;zero-valued \;element}{total \;number\;of;matrix\;elements\;(mxn)}\)
Sparsity(A) =
spars <- DocumentTermMatrix(js_train2) spars## <<DocumentTermMatrix (documents: 18400, terms: 1)>> ## Non-/sparse entries: 538/17862 ## Sparsity : 97% ## Maximal term length: 3 ## Weighting : term frequency (tf)scales::percent(17862 / (538+17862),.01)## [1] "97.08%"Apply the Naive Bayes classifier to original matrix and lower dimension matrix, what do you observe?
library(e1071) js_classifier <- naiveBayes(js_train2, js_train$sl) js_test_pred <- predict(js_classifier, js_test2) library(gmodels) jsp <- CrossTable(js_test_pred, js_test$sl)## ## ## Cell Contents ## |-------------------------| ## | N | ## | Chi-square contribution | ## | N / Row Total | ## | N / Col Total | ## | N / Table Total | ## |-------------------------| ## ## ## Total Observations in Table: 40 ## ## ## | js_test$sl ## js_test_pred | Low Stress | High Stress | Row Total | ## -------------|-------------|-------------|-------------| ## Low Stress | 29 | 5 | 34 | ## | 0.267 | 0.918 | | ## | 0.853 | 0.147 | 0.850 | ## | 0.935 | 0.556 | | ## | 0.725 | 0.125 | | ## -------------|-------------|-------------|-------------| ## High Stress | 2 | 4 | 6 | ## | 1.510 | 5.202 | | ## | 0.333 | 0.667 | 0.150 | ## | 0.065 | 0.444 | | ## | 0.050 | 0.100 | | ## -------------|-------------|-------------|-------------| ## Column Total | 31 | 9 | 40 | ## | 0.775 | 0.225 | | ## -------------|-------------|-------------|-------------| ## ##jsp## $t ## y ## x Low Stress High Stress ## Low Stress 29 5 ## High Stress 2 4 ## ## $prop.row ## y ## x Low Stress High Stress ## Low Stress 0.8529412 0.1470588 ## High Stress 0.3333333 0.6666667 ## ## $prop.col ## y ## x Low Stress High Stress ## Low Stress 0.93548387 0.55555556 ## High Stress 0.06451613 0.44444444 ## ## $prop.tbl ## y ## x Low Stress High Stress ## Low Stress 0.725 0.125 ## High Stress 0.050 0.100mod_TN <- jsp$prop.row[1, 1] mod_FP <- jsp$prop.row[1, 2] mod_FN <- jsp$prop.row[2, 1] mod_TP <- jsp$prop.row[2, 2] library(plotly)## Loading required package: ggplot2## ## Attaching package: 'ggplot2'## The following object is masked from 'package:NLP': ## ## annotate## ## Attaching package: 'plotly'## The following object is masked from 'package:ggplot2': ## ## last_plot## The following object is masked from 'package:stats': ## ## filter## The following object is masked from 'package:graphics': ## ## layoutplot_ly(x = c("TN", "FN", "FP", "TP"), y = c(mod_TN, mod_FN, mod_FP, mod_TP), name = c("TN", "FN", "FP", "TP"), type = "bar", color=c("TN", "FN", "FP", "TP")) %>% layout(title="Consusion Matrix", legend=list(title=list(text='<b> Metrics </b>')), xaxis=list(title='Metrics'), yaxis=list(title='Probability'))# True positive & True Negative are dominated with 85.2% & 66.6%. library(caret)## Loading required package: latticescales::percent(confusionMatrix(js_test_pred, js_test$sl)$overall[1],.01)## Accuracy ## "82.50%"#Accuracy of testing-data prediction is 82.5%Apply and compare LDA and Naive Bayes classifiers with respect to the error, specificity and sensitivity.
library("dplyr")## ## Attaching package: 'dplyr'## The following objects are masked from 'package:stats': ## ## filter, lag## The following objects are masked from 'package:base': ## ## intersect, setdiff, setequal, unionbinarizeFunction <- function(x) { ifelse(x=="Yes", 1,0) } # A function to Convert Categorical variables to numeric cat2Numeric <- function (dfInput) { df = as.data.frame(lapply( as.data.frame(dfInput), factor)) %>% mutate_all(binarizeFunction) return(df) } # define the numeric DF of predictors (X) and outcome (Y=stage) df_js_train = data.frame(cat2Numeric(js_train2), sl = js_train$sl) df_js_test = data.frame(cat2Numeric(js_test2), sl = js_test$sl) # Remove the multicollinearity via VIF assessment, library(car)## Loading required package: carData## ## Attaching package: 'car'## The following object is masked from 'package:dplyr': ## ## recodemcl<-data.frame(t(t(vif(lm(sl~.,data=df_js_train)))))## Warning in model.response(mf, "numeric"): using type = "numeric" with a factor ## response will be ignored## Warning in Ops.factor(y, z$residuals): '-' not meaningful for factors## Warning in Ops.factor(r, 2): '^' not meaningful for factors## Warning in cov2cor(v): diag(.) had 0 or NA entries; non-finite result is ## doubtfulcolnames(mcl) <- c("vif") mcl$index<-1:nrow(mcl) mcl2<-mcl%>%filter(vif<5)%>%select(2) list(mcl2$index)## [[1]] ## integer(0)df_js_train<- df_js_train[, c(1, 5, 6, 10, 11, 13, 14, 18, 20, 21, 23, 24, 27, 28, 29, 30, 31, 32, 34, 40, 41, 42, 43, 44, 47, 48, 52, 53, 54, 57, 58, 59, 61, 63, 64, 67, 69, 70, 74, 76, 77, 80, 90, 97, 101, 103, 105, 106, 107, 111, 112)] # Fit LDA set.seed(1234) js_lda <- MASS::lda(data=df_js_train, js_train$sl~.) js_pred = predict(js_lda, df_js_test) ###Comparison ## LDA CrossTable(js_pred$class, df_js_test$sl)## ## ## Cell Contents ## |-------------------------| ## | N | ## | Chi-square contribution | ## | N / Row Total | ## | N / Col Total | ## | N / Table Total | ## |-------------------------| ## ## ## Total Observations in Table: 40 ## ## ## | df_js_test$sl ## js_pred$class | Low Stress | High Stress | Row Total | ## --------------|-------------|-------------|-------------| ## Low Stress | 28 | 6 | 34 | ## | 0.103 | 0.356 | | ## | 0.824 | 0.176 | 0.850 | ## | 0.903 | 0.667 | | ## | 0.700 | 0.150 | | ## --------------|-------------|-------------|-------------| ## High Stress | 3 | 3 | 6 | ## | 0.585 | 2.017 | | ## | 0.500 | 0.500 | 0.150 | ## | 0.097 | 0.333 | | ## | 0.075 | 0.075 | | ## --------------|-------------|-------------|-------------| ## Column Total | 31 | 9 | 40 | ## | 0.775 | 0.225 | | ## --------------|-------------|-------------|-------------| ## ##confusionMatrix(js_pred$class, df_js_test$sl)$overall[1]## Accuracy ## 0.775## NB CrossTable(js_test_pred, js_test$sl)## ## ## Cell Contents ## |-------------------------| ## | N | ## | Chi-square contribution | ## | N / Row Total | ## | N / Col Total | ## | N / Table Total | ## |-------------------------| ## ## ## Total Observations in Table: 40 ## ## ## | js_test$sl ## js_test_pred | Low Stress | High Stress | Row Total | ## -------------|-------------|-------------|-------------| ## Low Stress | 29 | 5 | 34 | ## | 0.267 | 0.918 | | ## | 0.853 | 0.147 | 0.850 | ## | 0.935 | 0.556 | | ## | 0.725 | 0.125 | | ## -------------|-------------|-------------|-------------| ## High Stress | 2 | 4 | 6 | ## | 1.510 | 5.202 | | ## | 0.333 | 0.667 | 0.150 | ## | 0.065 | 0.444 | | ## | 0.050 | 0.100 | | ## -------------|-------------|-------------|-------------| ## Column Total | 31 | 9 | 40 | ## | 0.775 | 0.225 | | ## -------------|-------------|-------------|-------------| ## ##confusionMatrix(js_test_pred, js_test$sl)$overall[1]## Accuracy ## 0.825#LDA has an accurracy of 77.5% while NB has an accurracy of 82.5%. NB has lower Type I & Type II error to avoid overestimate stress level and avoid missing high stress respectively. ##Sensitity & Specificity #LDA confusionMatrix(js_pred$class, df_js_test$sl)$byClass[1:2]## Sensitivity Specificity ## 0.9032258 0.3333333#NB confusionMatrix(js_test_pred, js_test$sl)$byClass[1:2]## Sensitivity Specificity ## 0.9354839 0.4444444# NB has higher sensitity and specificity to correctly identify Low Stress and High Stress respectivity.